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ABSTRACT 



The sensitivity of the ATLAS experiment to the top-quark rare decay via flavor- 
changing neutral currents t ^ Zq {q represents c and u quarks) has been studied at 
A/i=14 TeV in two decay modes: 

1. The pure leptonic decay of gauge bosons: tt ZqWb jl^uh , (l=e, //). 

2. The leptonic decay of Z bosons and hadronic decay of W bosons: ti — > ZqWb — > 
l^l'jjjb , (l=e, /i). 

The dominant backgrounds Z + jets, WZ and tt have been analysed. The signal 
and backgrounds were generated via PYTHIA 5.7, and simulated and analysed using 
ATLFAST 2.14. A branching ratio for t Zq as low as 2.0x10"^ for the leptonic mode 
and S.OxlO"*^ for hadronic mode could be discovered at the 5a level with an integrated 
luminosity of 100 fb~^. 
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1 Introduction 



The existence of the top quark has been estabhshed at the Fermilab Tevatron by the 
CDF and DO Collaborations [1]. Due to the large value of the top quark mass, of order 
the Fermi scale, the top quark couples strongly to the electroweak symmetry breaking 
sector. If anomalous top quark couplings beyond the Standard Model (SM) exist, they 
could affect top quark production and decay processes at hadron and e+ e~ colliders 
[2,3]. For example, study of the flavour-changing neutral current top quark decay t — > Zq 
(where q represents either c or u quarks) is of great interest. As is well known [4,5], the 
LHC can be considered as a "top factory", producing about 80,000 ti events per day at 
L — 10^^ cm~^ s~^, making the LHC an ideal place to explore this rare decay. 

In this note, we present a study of the sensitivity of the ATLAS experiment to the 
branching ratio of the top-quark rare decay mode t ^ Zq {q = u,c). In the framework 
of the Standard Model, the loop suppression and heaviness of gauge bosons make this 
process extremely rare. The SM prediction for the branching ratio, of order 10~^^ [6], 
would render the decay unobservable even with the large samples of top quarks expected 
at the LHC. However, other models predict significantly larger branching ratios for this 
process. For example, in two-Higgs-doublet models, the value of Br(t — > Zq) can reach 
~ 10~^ [7], while Supersymmetric (SUSY) models (without i?-parity) can have even much 
higher values, Br(f Zq)r^ IQ-^ [8]. 

Experimentally, little is known about this decay. The only existing limit comes from a 
CDF analysis of its Runl data [9], yielding Br(t Zq) < 33% (95% CL). Given the SM 
prediction, an observation of this decay mode, even at the LHC, would provide a clear 
signal of new physics beyond the SM, such as new dynamical interactions of top quark, 
multi-Higgs doublets, exotic fermions or other possibilities [5,6,7]. In addition, this mode 
is of interest since, if the branching ratio were large, events of the form ti — > ZZ + cc 
could prove to be a serious background to events containing Z boson pairs and jets from 
cascade decays of squarks and gluinos [10]. 

The dominant mechanism for top quark production at the LHC is tt pair production 
via gg,qq ^ ti. The analyses presented here focus on two different final state topologies 
of ti -> ZqWb: 

1. The purely Icptonic decay of both gauge bosons: 
t^ Zq^ l+l-j ,i^Wb^ l^vh (l=e, ^i) 

2. The Icptonic decay of the Z boson and hadronic decay of the W boson: 
t^Zq^ l+l-j J^Wb^ jjb (l=e, //) 



2 



The value of the branching ratio of the second ( "hadronic" ) mode is three times higher 
than for the first ("leptonic") one. However, the hadronic mode suffers from enormous 
QCD-backgrounds, while the leptonic mode has a very distinct experimental signature. 
Both modes have been studied, and the results of each analysis will be presented below. 

2 Monte Carlo Event Generation 

The dominant source of top quark production at the LHC is pair-production via gg,qq —>■ 
tt. PYTHIA 5.7 was set up to produce tt events at ^/s = 14 TeV and rritop = 174 GeV, 
with proton structure functions CTEQ2L. Initial and final state QED and QCD (ISR, 
FSR) radiation, multiple interactions, fragmentations and decays of unstabled particles 
were enabled. 

The decay t Zq is not implemented in the standard release of PYTHIA. To include 
this process in PYTHIA, all individual decay channels of the top quark were first switched 
off, except for t Wb and t Ws. The channel t Ws was then replaced by the decay 
t Zq, by replacing W hj Z and s by c{u). The total cross-section for ti production 
was assumed to be cr^f =800 pb [12]. 

The analyses reconstructed the Z boson via its leptonic decay, Z . The experi- 

mental signature included, therefore, a pair of isolated charged leptons, as well as several 
jets. The SM backgrounds considered for this process were Z + jets production, WZ 
production, and ti — > WbWb production. 

Z + jets production at the LHC has a relatively large cross-section, dominated by 
qg — > Zq and qq Zg processes. To decrease the size of the background sample which 
needed to be generated, thresholds were imposed at the generator level on the invariant 
mass, m — V§ > 130 GeV, and transverse momentum, p± > 50 GeV, of the hard 
scattering process. The cross-section for this subsample of events was az+jets—^^^^ pb. 

The WZ background is the electroweak process pp — > W^Z + X, and has an assumed 
cross-section of awz — 26.58 pb. 

The performance of the ATLAS detector was simulated using the fast simulation pack- 
age ATLFAST 2.14 [11], which uses parametrizations of the detector resolution functions. 

3 The Leptonic Decay Mode 

The final state for the leptonic decay mode is ti — > ZqWb — > I'^l'jlvb. The experimental 
signature therefore includes three isolated charged leptons, two of which reconstruct a Z 
boson, and large missing transverse energy due to the neutrino. 
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The backgrounds considered were Z+jets production, followed by the decay Z l^l , 
pp — > W^Z + X — > + X, and tt W^hW~h l^vhl'vh. Background samples 

of 2.1 X WZ + jets events, 35700 WZ events, and 3.7 x 10*^ ti events were generated. 
Assuming the production cross-sections given earlier, and including the relevant branching 
ratios, these background samples correspond to an integrated luminosity of 100 fb"^. 

Table |l| summarizes the effects of the sequential application of the various analysis 
cuts on the background samples and on the sample of 20565 signal events of the topology 
tt ZqWb l+l-jlvb. 



Description 
of 

Cuts 


t Zc 
Signal 


Background Processes 


Z+jets 


z+w 


tt 


Nevt 


Eff (%) 


Nevt 


Eff (%) 


Nevt 


Eff (%) 


Nevt 


Eff (%) 


Nevt gen. 


20565 




2.1-10^ 




35000 




3.7-10^ 




Preselection 


16497 


80.2 


3.7-105 


1.7 


2941 


8.2 


8.5-105 


29.4 


3 leptons 


8885 


43.3 


945 


4.4-10-3 


1778 


5.0 


1858 


5.0-10-2 


Pt > 30 GeV 


6730 


32.7 


80 


3.7-10-^ 


1252 


3.5 


1600 


4.3-10-2 


2 jets 


4063 


19.8 


31 


1.5-10-^ 


225 


0.6 


596 


1.6-10-2 


Mz±6 GeV 


3450 


16.8 


24 


1.1-10-^ 


180 


0.5 


29 


7.8-10-^ 


one fe-tag 


1678 


8.2 


10 


4.7-10-5 


28 


0.04 


10 


2.7-10"^ 


Mi ± 8 GeV 


728 


3.5 








1 


2.8-10-3 


3 


8.1-10-5 


Mt ± 12 GeV 


973 


4.7 








1 


2.8-10-3 


4 


1.1-10"^ 


Mt ± 24 GeV 


1264 


6.1 








2 


5.6-10-3 


5 


1.3-10-^ 



Table 1: The numbers of events and efficiencies (%) of kinematic cuts applied in sequence 
for the signal and backgrounds for the leptonic mode. The numbers do not include the 
lepton identification efficiencies. 



Preselection cuts were first applied, requiring the presence of at least three charged 
leptons (electrons with > 5 GeV and muons with pr > Q GeV) within pseudorapidity 
\rj\ < 2.5. Of these, at least one pair of leptons must be of opposite sign and same flavour, 
compatible with them being produced from a Z decay. In addition, the number of jets in 
the event with pj:^ > 15 GeV was required to be at least two. The requirement of three 
leptons reduces significantly the Z + jets and tt backgrounds, while the requirement of 
two jets reduces significantly WZ and Z + jets backgrounds. 

The lepton criteria were then tightened, by requiring the presence of at least three 
isolated, charged leptons (electrons or muons) with > 20 GeV and |r/'| < 2.5. The 
isolation criterion required that there be no additional track with pt > 2 GeV in a cone 
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Figure 1: The px distributions of leptons which give a dilepton invariant mass within 
the accepted window around the Z mass, for leptonic mode signal and backgrounds, 
normahsed to unity. The arrow indicates the threshold value pip used for the analysis cut. 
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Figure 2: The pt distribution of the third lepton for the leptonic mode signal and back- 
grounds, normalised to unity. The arrow indicates the threshold value p^p used for the 
analysis cut. 
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of AR = y'(A0)2 + (At/)^ = 0.3 around the lepton direction. Fig. 1 presents the pr 
distribution of the two leptons consistent with originating from a Z decay for signal and 
backgrounds, while Fig. 2 shows the px distribution for the third lepton. Tightening the 
lepton criteria significantly reduces the Z + jets and tt backgrounds, where any third 
lepton found typically comes from cascade decays of 6-jets. 

The next requirement, namely that the missing transverse momentum in the event 
satisfies p^'** > 30 GeV, is effective at further reducing the Z + jets background (see Fig. 
3), while having little impact on the signal and other background sources. 
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Figure 3: The reconstructed pfp^^^ distributions for leptonic mode signal and backgrounds, 
normalised to unity. The arrow indicates the threshold value used for the kinematic cut. 

Next, it was demanded that there be at least two jets with p^f* > 50 GeV, \rf^^\ < 
2.5, and satisfying the following isolation conditions: ARjj > 0.4 (jet-jet isolation) and 
ARij > 0.4 (lepton-jet isolation). Fig. 4 presents the number of jets with p^^^* > 50 GeV 
in an event. One can see that the cut requiring the presence of two and more such jets in 
each event effectively suppresses the WZ background. 

The presence of a reconstructed Z — > decay is a powerful cut against the tt 
background. A like-sign, same-flavor pair of isolated leptons was required to reconstruct 
to the Z mass within a window of Mz ± 6 GeV mass window. Fig. 5(a) presents the 
distribution of reconstructed invariant mass of // pairs ma, for all dilepton combinations 
for the signal events. The width of the accepted window corresponds to approximately 
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Figure 4: Distribution of jet multiplicity (threshold at pl^^ > 50 GeV) for signal and 
backgrounds, normalised to unity. The arrow indicates the threshold value of the number 
of jets used for the kinematic cut. 

twice the Z mass resolution of about 2.9 GeV. 

The next requirement was the presence in the event of one tagged 6-jet, which is 
effective at further reducing the WZ background (see Fig. 6). 

Finally, a peak at the top quark mass in the Zj invariant mass distribution was sought. 
In Fig. 5(b), the distribution of reconstructed invariant mass muj for all combinations 
of llj is presented for the signal events. The top quark mass resolution is a{miij) = 14 
GeV. Accepted combinations were required to lie within a window around the known top 
quark mass. As a check of the stability of the results, three different mass window were 
considered: mzj = ± 8 GeV (narrow cut), nit ± 12 GeV (~ a), and nit ±24 GeV 
(~ 2a). The top mass window removes almost completely the remaining background. 

As summarized in Table ^ the final signal efficiency was 6.1% for the nit ±24 GeV 
mass window, with a total background of 7 events for an integrated luminosity of 100 fb~^. 
The results in the table do not yet include the lepton identification efficiencies. Assuming 
an efficiency of 90% for each of the three leptons, the results correspond to a 5cr discovery 
potential for Br(t Zq) as low as 2.0x10""^ and for an integrated luminosity of 100 fb~^. 
This result is in good agreement with that of reference [13]. 
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Figure 5: (a) Distribution of reconstructed invariant mass of the lepton pairs, mu for the 
leptonic mode, (b) Distribution of reconstructed invariant mass oi t ^ Zq ^ llj for the 
leptonic mode. 

4 The Hadronic Decay Mode 

The final state for the hadronic W decay mode is tt ZcWb — > l~^l~jjjb. The experi- 
mental signature for the hadronic mode includes, therefore, two isolated charged leptons 
which originate from the Z decay, and four energetic jets. 

This mode has the following backgrounds: II) + jets, pp W^Z + X ^ 

+ X, and ti WbWb with the final state topologies (a) l^ubl'ub, (b) jjbjjb, or 
(c) l^ubjjb. In the case of (a), the additional two jets must come from QCD radiation, 
while in (b) and (c) the source of leptons is from cascade decays. Therefore, requiring 
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Figure 6: Distribution of 6-jet multiplicity (threshold at P^,^* > 50 GeV) for signal and 
backgrounds, normalised to unity. 

two isolated energetic leptons and four energetic jets, among which only one is tagged 
as a 6-jet, significantly suppresses the tt background. It has been checked that the small 
remaining ti background is removed by the reconstruction cuts asking for mu and muj 
to lie within windows around mz and rrit respectively. Therefore, the ti background is 
reduced to a negligible level, and is not considered further. 

Background samples of 2.1 x lO^Z + jets events, and 1.2 x lO^WZ events were gen- 
erated. Assuming the production cross-sections given earlier, and including the relevant 
branching ratios, these background samples correspond to an integrated luminosity of 100 
fb-i. 

Table |^ summarizes the effects of the sequential application of the various analysis 
cuts on the background samples and on the sample of 19000 signal events of the topology 
tt ZqWb l+l'jjjb. 

The analysis began with preselection cuts requiring that the event contains at least 
two charged leptons (electrons with px > 5 GeV within pseudorapidity lijl < 2.5 and 
muons with Pt > 6 GeV within pseudorapidity lijl < 2.4), and include a pair of opposite- 
sign and same-flavour leptons, compatible with them having come from a Z decay. In 
addition, the number of jets with p:^* > 15 GeV was required to be at least four. After 
preselection cuts, 46% of the signal events are accepted, while only 3.5% and 4.1% of the 
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Z + jets and WZ background events, respectively, are retained. 

The next cuts required the presence of two isolated leptons with > 20 GeV and 
\rf\ < 2.5, and the demand for at least four jets with Pj^*^* > 50 GeV and \r]^ < 2.5. 
Jet-jet and lepton-jet isolation criteria were then applied, as was done in the leptonic 
mode analysis. 

A cut was then placed on the dilepton invariant mass, requiring that it lie within a 
window of ±6 GeV around ruz, the same mass window as used for the leptonic mode. 
Fig. 7(a) presents the distribution of reconstructed dilepton invariant mass for the signal 
sample. 

To suppress the large remaining Z + jets background, it was necessary to use the 
information that signal events contain, in addition to the decay t Zq, a hadronic decay 
t Wb jjb of the other top quark. The hadronic top quark decay was, therefore, 
reconstructed as part of the signal requirement. First, a pair of jets was required to 
have an invariant mass rrijj with a window of ±16 GeV around mw- Fig. 8. a shows the 
distribution of reconstructed rrijj for the best combinations of jj for the signal events. 
The W mass resolution is am^j = 8 GeV. 

The requirement was then made to have one jet tagged as a 6-jet. Finally, the jjb 
invariant mass was required to lie within a window of ±8 GeV around m^. Fig. 8(b) 
presents the distribution of the reconstructed invariant top mass {mjjb) for the best com- 
binations of jjb for the signal. The top mass resolution is a{mjjb) — 18.5 GeV, implying 
that the mass window applied is rather narrow in order to get a large background rejec- 
tion. The sequence of cuts required to reconstruct the hadronic decay of the other top 
quark dramatically suppresses the background, but also reduces the signal efficiency by 
almost an order of magnitude. 

In Fig. 7.b the distribution of reconstructed t^Zq invariant top mass muj for the 
best combinations of II j is presented for the signal. The resolution a of muj distribution 
is ajnuj — 9.9 GeV. For the reconstruction of the top mass for the decay t— >Zq, the same 
mass windows have been chosen as for the leptonic mode: mzq ±8 GeV (narrow cut), 
mzq ±12 GeV (~ a) mzq ±24 GeV (~ 2a). 

It is worth mentioning that, in the mass window ruzq ±8 GcV, 91% of the accepted 
events contain c-jets, t— lie and only 10% of events are reconstructed with light jets. By 
widening the mass window, the mixture of events reconstructed with light jets increases 
from 10% to 22%, but the number of events reconstructed with c-jets increases too. 

The analysis cuts reduce the WZ background to a negligible level in three Zj mass 
windows. The Z + jets background vanishes in mass window mzq ±8 GeV, while one 
event is accepted in mzq ±12 GeV, and two events in mzq ±24 GeV. 
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Figure 7: a) Distribution of reconstructed invariant mass of the lepton pairs, mu for the 
best combination (hadronic mode), b) Distribution of reconstructed invariant mass of 
t II j for the best combination of II j (hadronic mode). 

The results in Table |] imply that a value of Br(t ^ Zq ) as low as 5.9x10"^ could be 
discovered at the 5a level with an integrated luminosity 100 fb~^. 



5 Conclusions 

We have studied the ATLAS sensitivity to the FCNC top quark rare decay t Zq 
(g = u, c) for an integrated luminosity of 100 fb^^. 

As summarized in Table ^, the results demonstrate that, in the leptonic mode, a 
branching ratio as low as 2.0x10"^ and, in the hadronic mode, as low as 5.9x10"^ could 
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Figure 8: a) Distribution of reconstructed invariant mass of the jet pairs, rrijj for the best 
combination (hadronic mode), b) Distribution of reconstructed invariant mass of t jjb 
for the best combination of jjb (hadronic mode). 

be discovered at the 5a level with an integrated luminosity of 100 fb~^. 

Table § compares the relative efiiciences of some of the important kinematic cuts for 
leptonic and hadronic modes of the signal. One can see that the relative efficiences of 
leptons, jets, Z mass, 6-jet and top mass cuts for leptonic and hadronic modes are in good 
agreement, as expected. This agreement supports the consistency of the two analyses, 
despite different versions of the ATLFAST code being used in the two cases. 

Combining the results from the two decay modes extends the 5a discovery potential 
down to a branching ratio as low as 1.8x10"^. 
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Description 
of 
Cuts 


t Zc 
Signal 


Background Processes 


Z+jets 


z+w 


Nevt 


Eff (%) 


Nevt 


Eff (%) 


Nevt 


Eff (%) 


Nevt gen. 


19002 




2.1-10^ 




1.2-10^ 




Pros(4ecti()ii 


8712 


16.0 


7.5- K/' 


3.5 


1970 


1.1 


2 le])t()iis 


7171 


3 / . / 


5.9-10" 


2.8 


1156 


3.7 


1 jots 


2896 


15.2 


63178 


0.29 


100 


0.3 


ARjj > 0.4 


2828 


14.9 


60421 


0.28 


390 


0.3 


ARij > 0.4 


2826 


14.9 


60394 


0.28 


361 


0.3 


Mz±6 GeV 


2426 


12.8 


50973 


0.24 


268 


0.2 


Mw ± 16 GeV 


1006 


5.3 


14170 


6.6-10-2 


139 


0.1 


one 6-tag 


432 


2.2 


1379 


6.4-10-=^ 


11 


9.1-10-3 


Mxvi, = Ml ± 8 G( V 


106 


0.6 


90 


1.2-10-^ 


1 


8.3- 10^^ 


Mz„ = Ml ± 8 G( V 


16 


0.2 














Mzg = Mt ± 12 GeV 


58 


0.3 


1 


4.8-10-6 








Mzq = Mt ± 24 GeV 


74 


0.4 


2 


9.3-10-6 









Table 2: The numbers of events and efficiencies (%) of selection cuts applied in sequence 
for the signal in the hadronic decay mode. 



Window 
for Mzg 


Sensitivity to Br(t— >Zq) 


Leptonic Mode 


Hadronic Mode 


Combined Result 


Mt±8 GeV 


2.80-10-^ 


9.60-10-^ 


2.60-10-^ 


Mt ± 12 GeV 


2.34-10-^ 


6.70-10-^ 


2.18-10-^ 


Mt ± 24 GeV 


2.01-10-^ 


5.86-10-^ 


1.80-10-^ 



Table 3: Summary of the results. 
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CUTS 


LEPTONIC MODE 


HADRONIC MODE 


Rel.Effic. (%) 


Rel.Effic. (%) 


31 with P| > 20 GeV in \r]^\ < 2.5 


70.1 




21 with P| > 20 GeV in \r]^\ < 2.5 




82.0 


2 isolated jets with Pf'* > 50 GeV 


48.6 




4 isolated jets with P^'* > 50 GeV 




39.4 


Lepton-jets isolation: ARij > 0.4 


94.0 


99.0 


Z mass Mz ± 6 GeV 


84.9 


85.8 


one tagged jet in the event 


48.6 


42.9 


t^Zq mass : Mzq ± 8 GeV 


43.4 


43.4 


t^Zq mass : Mzg ± 12 Gel^ 


49.2 


54.7 


t^Zq mass : Mzq ± 24 GeF 


75.3 


69.8 



Table 4: The relative efficiencies (%) of some of the kinematic cuts for leptonic and 
hadronic modes of the signal. 
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